A Patient-Specific Mixed-Reality Visualization Tool for Thoracic Surgical Planning.

PURPOSE: Identifying small lung lesions during minimally invasive thoracic surgery can be challenging. We describe 3-dimensional mixed-reality visualization technology that may facilitate noninvasive nodule localization. DESCRIPTION: A software application and medical image processing pipeline were developed for the Microsoft HoloLens to incorporate patient-specific data and provide a mixed-reality tool to explore and manipulate chest anatomy with a custom-designed user interface featuring gesture and voice recognition. EVALUATION: A needs assessment between engineering and clinical disciplines identified the potential utility of mixed-reality technology in facilitating safe and effective resection of small lung nodules. Through an iterative process, we developed a prototype employing a wearable headset that allows the user to (1) view a patient's original preoperative imaging; (2) manipulate a 3-dimensional rendering of that patient's chest anatomy including the bronchial, osseus, and vascular structures; and (3) simulate lung deflation and surgical instrument placement. CONCLUSIONS: Mixed-reality visualization during surgical planning may facilitate accurate and rapid identification of small lung lesions during minimally invasive surgeries and reduce the need for additional invasive preoperative localization procedures.

Factored Occlusion: Single Spatial Light Modulator Occlusion-capable Optical See-through Augmented Reality Display.

Occlusion is a powerful visual cue that is crucial for depth perception and realism in optical see-through augmented reality (OST-AR). However, existing OST-AR systems additively overlay physical and digital content with beam combiners - an approach that does not easily support mutual occlusion, resulting in virtual objects that appear semi-transparent and unrealistic. In this work, we propose a new type of occlusion-capable OST-AR system. Rather than additively combining the real and virtual worlds, we employ a single digital micromirror device (DMD) to merge the respective light paths in a multiplicative manner. This unique approach allows us to simultaneously block light incident from the physical scene on a pixel-by-pixel basis while also modulating the light emitted by a light-emitting diode (LED) to display digital content. Our technique builds on mixed binary/continuous factorization algorithms to optimize time-multiplexed binary DMD patterns and their corresponding LED colors to approximate a target augmented reality (AR) scene. In simulations and with a prototype benchtop display, we demonstrate hard-edge occlusions, plausible shadows, and also gaze-contingent optimization of this novel display mode, which only requires a single spatial light modulator.

Handheld volumetric manual compression-based quantitative microelastography.

Compression optical coherence elastography (OCE) typically requires a mechanical actuator to impart a controlled uniform strain to the sample. However, for handheld scanning, this adds complexity to the design of the probe and the actuator stroke limits the amount of strain that can be applied. In this work, we present a new volumetric imaging approach that utilizes bidirectional manual compression via the natural motion of the user's hand to induce strain to the sample, realizing compact, actuator-free, handheld compression OCE. In this way, we are able to demonstrate rapid acquisition of three-dimensional quantitative microelastography (QME) datasets of a tissue volume (6 × 6 × 1 mm3 ) in 3.4 seconds. We characterize the elasticity sensitivity of this freehand manual compression approach using a homogeneous silicone phantom and demonstrate comparable performance to a benchtop mounted, actuator-based approach. In addition, we demonstrate handheld volumetric manual compression-based QME on a tissue-mimicking phantom with an embedded stiff inclusion and on freshly excised human breast specimens from both mastectomy and wide local excision (WLE) surgeries. Tissue results are coregistered with postoperative histology, verifying the capability of our approach to measure the elasticity of tissue and to distinguish stiff tumor from surrounding soft benign tissue.

Handheld probe for quantitative micro-elastography.

Optical coherence elastography (OCE) has been proposed for a range of clinical applications. However, the majority of these studies have been performed using bulky, lab-based imaging systems. A compact, handheld imaging probe would accelerate clinical translation, however, to date, this had been inhibited by the slow scan rates of compact devices and the motion artifact induced by the user's hand. In this paper, we present a proof-of-concept, handheld quantitative micro-elastography (QME) probe capable of scanning a 6 × 6 × 1 mm volume of tissue in 3.4 seconds. This handheld probe is enabled by a novel QME acquisition protocol that incorporates a custom bidirectional scan pattern driving a microelectromechanical system (MEMS) scanner, synchronized with the sample deformation induced by an annular PZT actuator. The custom scan pattern reduces the total acquisition time and the time difference between B-scans used to generate displacement maps, minimizing the impact of motion artifact. We test the feasibility of the handheld QME probe on a tissue-mimicking silicone phantom, demonstrating comparable image quality to a bench-mounted setup. In addition, we present the first handheld QME scans performed on human breast tissue specimens. For each specimen, quantitative micro-elastograms are co-registered with, and validated by, histology, demonstrating the ability to distinguish stiff cancerous tissue from surrounding soft benign tissue.

Handheld optical palpation of turbid tissue with motion-artifact correction.

Handheld imaging probes are needed to extend the clinical translation of optical elastography to in vivo applications, yet such probes have received little attention. In this paper, we present the first demonstration of optical palpation using a handheld probe. Optical palpation is a variant of optical elastography that uses three-dimensional optical coherence tomography (3D-OCT) to provide maps of stress at the tissue surface under static compression. Using this technique, stiff features present beneath the surface of turbid tissues are identified, providing mechanical contrast complementary to the optical contrast provided by OCT. However, during handheld operation, relative motion between the probe and the tissue can induce motion artifact, causing spatial distortion of 3D-OCT and in turn, optical palpation images. We overcome this issue using a novel, dual-function bi-layer that provides both a fiducial marker for co-registration and a compliant section for estimation of the stress at the tissue surface. Co-registration of digital photographs of the bi-layer laid out over the tissue surface is used to measure and correct for motion in the lateral (xy) plane. We also demonstrate, for the first time, that optical palpation can be used as a method for monitoring pressure applied to the tissue during handheld operation, thus providing a more repeatable and robust imaging technique between different users. Handheld optical palpation is demonstrated on a structured phantom, in vivo human skin and excised human breast tissue. In each case, image quality comparable to bench-top 3D-OCT and optical palpation is achieved.